• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    Haptic augmented reality device to facilitate actions on a refueling pole in flight. Haptic augmented reality device to facilitate the actions of control of the pole in mid-flight, allowing control over the boom with one hand, without the need for any training as far as their movements are concerned. The system, in addition, not only provides information on how the operation is being performed, but also provides assistance, indicating to the "boomer" when a certain movement is totally prohibited (making such movement impossible) or not recommended (making it necessary to apply more strength to do it). (Machine-translation by Google Translate, not legally binding)
    公开号:ES2580252A1
    申请号:ES201531426
    申请日:2015-10-05
    公开日:2016-08-22
    发明作者:Alberto ADARVE LOZANO
    申请人:Defensya Ingenieria Internacional SL；
    IPC主号:
    专利说明:

    OBJECT OF THE INVENTION
    It is the object of the present invention, as the title of the invention establishes, a haptic augmented reality device to facilitate actions on a pole
    10 refueling in flight from a nurse or tanker to a ship to refuel.
    Also object of the invention is a device for controlling the haptic augmented reality device.
    Another object of the invention is the system in which the control device of the haptic augmented reality device is integrated.
    When refueling tasks are performed in flight from a nurse aircraft to a receiving aircraft, a pole is used that departs from the nurse aircraft and is provided
    20 at its end, with what is known in the sector as "boom", while the specialists in charge of managing the boom are known as "boomers" or pole operators, using for this purpose joysticks called "joysticks".
    25 Characterizes the haptic augmented reality device, as well as the control device in which it is integrated, the fact of allowing control actions on the “boom” and the pole with only one hand, without the need for training in what He refers to his movements. In addition, the system provides information and help on how the operation is being carried out, indicating the operator without an operation.
    30 is recommended or not.
    Therefore, the present invention is circumscribed within the scope of haptic control systems on the one hand, and in-flight refueling control operation systems.
    BACKGROUND OF THE INVENTION


    Currently, when refueling tasks are performed in flight with a pole or boom, the specialists in charge of handling them are known as boomers or operators, using joystick devices or joysticks, for control of that resupply element.
    5 Said control elements allow the movement of said Boom vertically or horizontally in a manner similar to that used in video games to fly in an airplane, or drive a car. Similarly, the extraction and retraction of the boom or extendable part of the boom is done with another joystick or joystick that is usually
    10 drive with your left hand. Although these devices are quite intuitive, prior training is required for the operator to associate their movements with those of the boom to be controlled. As indicated, they require both hands to handle the refueling boom and do not usually provide the user with any type of feedback on how the operation is taking place or on the
    15 actions that are allowed or not on those recommended during a refueling. The control levers or joystick of the state of the art used for the management of the poles or booms, are complex to handle, having to use both hands, in addition to requiring a previous training, lacking any means of
    20 indication of whether the movement to be performed is allowed or not.
    Therefore, it is the object of the present invention to develop a device that facilitates the control and handling of the poles or boom used in refueling in flight, which can be used with only one hand and that has means of indicating whether the
    25 movements that are intended to be carried out are allowed or not, developing a device such as the one described below and which is essentially included in the first claim.
    DESCRIPTION OF THE INVENTION
    30 The object of the present invention is a haptic augmented reality device to facilitate the control actions of the pole in full flight, allowing control over the boom with one hand, without the need for any training as far as its movements are concerned. refer. The system also not only provides
    35 information on how the operation is being performed, but also provides help, telling the "boomer" when a given movement is fully


    prohibited (preventing such movement) or is not recommended (making it necessary to apply more force to perform it).
    Thanks to the device it is possible to completely or partially automate the process of
    5 refueling in flight. This device and the associated control mechanism improveradically the current situation, allowing the user to handle with one hand theboom
    Also object of the present invention is the control device in which
    10 integrates the augmented reality device that facilitates the actions on a refueling pole.
    The haptic device and the control device are integrated into a system that helps control and refuel in flight.
    15 The haptic device comprises:
    - a first double set of motors that allow a horizontal displacement similar to a real boom
    - a second double set of motors that allow a pitching movement or
    20 of "pitch", and which rotate with respect to an axis orthogonal to the axis of rotation of the first set of motors.
    - a counterweight to balance forces
    - a vibrator that depending on the type and frequency of it may indicate
    user the existence of any inconvenience or not at the time of making an intended movement.
    A position sensor or decoder has been included in all engines in order to properly determine the position of the motor with respect to its axis of rotation.
    30 The system in which the haptic control device is integrated in turn comprises:
    - Information subsystem comprising:
    o Database with 3D models of all the necessary elements in a refueling operation (poles, airplanes, etc.).
    35 o Expert system including information on the operation (prohibited movements, recommendations, etc.).


    - Perception subsystem comprising:
    o Means for capturing images. Redundant system in charge of
    visual capture of the information related to the operation. System 5 is prepared to merge various types of visual information.
    • Image capture with structured lighting: a sequence of images captured with a set of cameras at a predetermined rate of frames per second.
    • Image capture with cameras with 10-flight time sensors (ToF).
    • Camera information of the standard instrumentation of refueling aircraft. The system allows the connection of the cameras located as standard in these aircraft, suitable for the initial phase of the recharge operation, when the
    15 aircraft to refuel is more than 15 meters away. This information will be used to provide additional redundancy to the system.
    o Collection of contextual information. Sensor system that captures information of the environment of interest for the operation (GPS Position of
    20 reception aircraft, GPS position of reload aircraft, relative distance pole-receptacle humidity, temperature, wind, real pole motion sensors, etc.).
    - Analysis Subsystem It is the fundamental part of the invention, and allows
    25 create a 3D model of the status of the operation in real time (maximum of 2 delay frames,) from the analysis of the data provided by the perception system. Specifically, using image analysis methods with structured lighting on a combination of the captured images and the information of the location of the receiving aircraft, allows to perform a
    30 virtual positioning of a 3D model of the receiver in the boom environment.
    - Refueling intelligence subsystem. From the result of the analysis performed by the previous subsystem, and using the information of the expert system in refueling operations, this subsystem determines and updates
    35 real time:


    o The recommended sequence of movements to perform to successfully carry out the information.
    o The set of movements to prevent (decomposed into actions).
    o The set of movements to be hindered, and to what degree (decomposed5 in shares).
    - Remote control haptic subsystem as described above
    - Feedback Subsystem Its mission is to produce in the subsystem
    10 remote control haptic actions that reflect both the effect of the environment (e.g. the force of the wind on the system), and the normative and recommended strategies for performing the operation (expert refueling operating system). For that reason we distinguish 2 parts:
    • Pole feedback - Haptic subsystem. A set of
    15 sensors in the real pole capture the forces to which the real pole is subjected and convert these forces into actions on the control motors of the haptic subsystem.
    • Refueling intelligence subsystem feedback - Haptic subsystem. The refueling intelligence subsystem
    20 determines in real time the set of movements to favor and hinder and to what degree and translates the set into a series of actions on the control engines of the haptic subsystem.
    Unless otherwise indicated, all technical and scientific elements used in
    The present specification has the meaning that the normal expert in the art to which this invention belongs belongs to. In the practice of the present invention procedures and materials similar or equivalent to those described herein can be used.
    Throughout the description and the claims the word "comprises" and its variants are not intended to exclude other technical characteristics, additives, components or steps. For those skilled in the art, other objects, advantages and features of the invention will be derived partly from the description and partly from the practice of the invention.
    EXPLANATION OF THE FIGURES


    To complement the description that is being made and in order to help a better understanding of the characteristics of the invention, according to a preferred example of practical realization thereof, it is accompanied as an integral part of
    5 said description, a set of drawings in which the following is represented as illustrative and not limiting.
    In Figure 1, we can see a representation of the haptic augmented reality device object of the request.
    10 In Figure 2 we can see the control device in which the haptic augmented reality device is integrated.
    PREFERRED EMBODIMENT OF THE INVENTION.
    In view of the figures, a preferred embodiment of the proposed invention is described below.
    In figure 1 we can see how the haptic device comprises: 20 - a pole modeling or prototype (1)
    - control and drive means (2) from which a transmission arm (3) emerges from the drive to the modeling of the pole (1).
    Where the control and drive measures (2) comprise: 25 - a first double set of motors (4) that allow horizontal movement of the pole modeling (1).
    - A second double set of motors (5) that allow a pitching or "pitch" movement, and which rotate with respect to an axis orthogonal to the axis of rotation of the first set of motors.
    30 -A counterweight to balance the forces of both engine sets.
    - First electronic means (7) for driving and controlling the motors (4) and (5).
    The modeling of the pole (2) has a shape similar to the real pole that is intended to be handled and which comprises:


    - a few second electronic means (8) that control the elements that are part of the modeling of the pole
    - a vibrator (9) housed inside the pole modeling (8)
    -a first button (10), a second button (11) and a third button (12),5 -a acceleration sensor (13), and
    - a magnet (14).
    -One LEDs (15).Allowing all of them control over the boom with one hand.
    10 Figure 2 shows the control device (21) in which the haptic device shown in Figure 1 is integrated, and comprising:
    - a housing (16) inside which the drive and control means (2) of the haptic device are housed and from which the pole modeling (1) emerges.
    15 -A armrest (17) placed under the housing (16) and attached to it by vertical bars (18), leaving a free space through which to pass and support an arm while the hand takes the modeling of the pole (1) .
    - A base (18) arranged under the armrest (17) and which is provided with a position gauge (20) and an automatic connector (19).
    20 Describing sufficiently the nature of the present invention, as well as the way of putting it into practice, it is stated that, within its essentiality, it may be implemented in other embodiments that differ in detail from that indicated by way of example, and which will also achieve the protection that
    25 collects, as long as it does not alter, change or modify its fundamental principle.

    权利要求:
    Claims (2)
    [1]
    1.- Pole control device for refueling operations by means of a pole, with touch control and reception, based on haptic augmented reality technology comprising:
    - a pole modeling or prototype (1)
    - control and drive means (2) from which a transmission arm (3) emerges from the drive to the modeling of the pole (1).
    Characterized by the control and actuation measures (2) comprising:
    - a first double set of motors (4) that allow horizontal movement of the pole modeling (1).
    - A second double set of motors (5) that allow a pitching or "pitch" movement, and which rotate with respect to an axis orthogonal to the axis of rotation of the first set of motors.
    - A counterweight to balance the forces of both engine games.
    - First electronic means (7) for driving and controlling the motors (4) and (5),
    The modeling of the pole (2) has a shape similar to the real pole that is intended to be handled and which comprises:
    - a few second electronic means (8) that control the elements that are part of the modeling of the pole
    - a vibrator (9) housed inside the pole modeling (8)
    - a series of buttons (10), (11) and (12),
    - an acceleration sensor (13), and
    - a magnet (14).
    - Some LEDs (15). Allowing all of them control over the boom with one hand
    [2]
    2. Ergonomic and support assembly of the pole control device for refueling operations by means of a pole, with touch control and reception, based on haptic augmented reality technology, defined in claim 1 characterized in that it comprises:

    - a housing (16) inside which the drive and control means (2) of the haptic device are housed and from which the pole modeling (1) emerges.
    - An armrest (17) placed under the housing (16) and attached to it by
    5 vertical bars (18), leaving a free space to pass and support an arm while the hand takes the modeling of the pole (1).
    - A base (18) arranged under the armrest (17) and which is provided with a position gauge (20) and an automatic connector (19).
    10 3. System of actuation and control of a refueling assembly in flight by means of a pole, in which the pole control device is integrated, with touch control and reception, based on haptic augmented reality technology, defined in claim 1 , characterized in that it comprises:
    - An information subsystem comprising:
    o Database with 3D models of all the necessary elements in a refueling operation (poles, airplanes, etc.).
    o Expert system including information on operation 20 (prohibited movements, recommendations, etc.).
    - A Perception subsystem comprising:
    o Means of Image Capture, which is a redundant system in charge of the visual capture of the information related to the operation. 25 Means of capturing contextual information by means of a system of sensors that capture the information of the environment of interest for the operation, being some or several of the following (GPS position of reception aircraft, GPS position of reload aircraft, relative distance moisture pole-receptacle,
    30 temperature, wind, real pole motion sensors, etc.). .
    - An Analysis subsystem that allows you to create a 3D model of the state of the
    real-time operation, 35

    - A refueling intelligence subsystem, in which, based on the result of the analysis performed by the previous subsystem, and using information from the expert system of refueling operations, this subsystem determines and updates in real time:
    5 o The recommended sequence of movements to perform to successfully carry out the information.
    o The set of movements to prevent (decomposed into actions).
    o The set of movements to be hindered, and to what degree (decomposed
    in actions). 10
    - A subsystem comprising the pole control device, with touch control and reception, based on haptic augmented reality technology, defined in claim 1,
    -
    15 - A feedback subsystem whose mission is to produce in the remote control haptic subsystem actions that reflect both the effect of the environment (eg wind force on the system), and the normative and recommended strategies for performing the operation (expert system of refueling operation).
    20. System according to claim 3 characterized in that the visual collection means can merge several types of visual information.
    • Image capture with structured lighting: a sequence of
    images captured with a set of cameras at a predetermined rate of 25 frames per second.
    • Image capture with cameras with flight time sensors (ToF).
    • Cameras information of the standard instrumentation of refueling aircraft. Our system allows the connection of cameras located as standard in these aircraft, suitable for the initial phase of the operation
    30 recharge, when the aircraft to refuel is more than 15 meters away. This information will be used to provide additional redundancy to the system.
    35. System according to claim 4 characterized in that it comprises the parts of:

    - Pole feedback - haptic subsystem comprising a set of sensors in the real pole captures the forces to which the real pole is subjected and converts these forces into actions on the control engines of the haptic subsystem.
    • Refueling intelligence subsystem feedback - haptic subsystem that determines in real time the set of movements to favor and hinder and to what degree and translates the set into a series of actions on the control engines of the haptic subsystem.



    类似技术:
    公开号 | 公开日 | 专利标题
    US10648809B2|2020-05-12|Adaptive compass calibration based on local field conditions
    CN105492985B|2019-06-04|A kind of system and method for the control loose impediment in environment
    CN104769496B|2019-03-15|Flight video camera with the rope component for positioning and interacting
    CN105981074B|2018-02-02|For demarcating system, the method and apparatus of imaging device
    US20190241263A1|2019-08-08|Flight path generation method, flight path generation system, flight vehicle, program, and storage medium
    CN107531322A|2018-01-02|Aerial capture platform
    CN106796761A|2017-05-31|System and method for supporting simulation mobile
    US20160327389A1|2016-11-10|Calibration Transfer Between Two Devices
    JPWO2018193574A1|2020-02-27|Flight path generation method, information processing apparatus, flight path generation system, program, and recording medium
    CN106814735B|2020-08-14|Control system of unmanned ship
    US20200304719A1|2020-09-24|Control device, system, control method, and program
    ES2580252B2|2017-03-17|HAPPY INCREASED REALITY DEVICE TO FACILITATE ACTIONS ON A FLIGHT FILLING LOSS
    US20200133097A1|2020-04-30|Control device, moving body, control method and procedure
    ES2757678T3|2020-04-29|Time-phase imaging for an artificial point of view
    US10852723B2|2020-12-01|Unmanned aerial vehicle swarm photography
    CN107077144B|2021-05-25|Method, system and control device for displaying state of movable device
    CN108450032B|2021-03-02|Flight control method and device
    EP3689754A1|2020-08-05|Flying body, living body detection system, living body detection method, program and recording medium
    CN107318268B|2020-07-17|Flight control method, device, control terminal, flight system and processor
    CN112882645A|2021-06-01|Channel planning method, control end, aircraft and channel planning system
    JPWO2021130980A1|2021-12-23|Flight route display method and information processing device for the aircraft
    JPWO2021064982A1|2021-11-04|Information processing device and information processing method
    CN108780325B|2022-03-18|System and method for adjusting unmanned aerial vehicle trajectory
    US20210034052A1|2021-02-04|Information processing device, instruction method for prompting information, program, and recording medium
    JP2020170213A|2020-10-15|Drone-work support system and drone-work support method
    同族专利:
    公开号 | 公开日
    EP3361349A4|2019-06-12|
    WO2017051056A1|2017-03-30|
    EP3361349B1|2021-07-21|
    US20180290308A1|2018-10-11|
    CN108351686A|2018-07-31|
    AU2016328885A1|2018-04-12|
    US11054931B2|2021-07-06|
    EP3361349A1|2018-08-15|
    ES2580252B2|2017-03-17|
    CN108351686B|2021-04-16|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20080302916A1|2005-03-24|2008-12-11|Speer Thomas E|Systems and methods for automatically and semiautomatically controlling aircraft refueling|
    US20080142642A1|2006-10-26|2008-06-19|Honeywell International, Inc.|Cogless motor driven active user interface haptic feedback system|
    US20100200705A1|2009-02-12|2010-08-12|Eads Construcciones Aeronauticas, S.A.|Control and command assembly for aircraft|
    EP2280359A1|2009-07-31|2011-02-02|EADS Construcciones Aeronauticas, S.A.|Training method and system using augmented reality|
    US20140214206A1|2011-10-26|2014-07-31|Cambridge Surgical Instruments, Inc.|Device for providing tactile feedback for robotic apparatus using actuation|
    FR2643502B1|1989-02-20|1996-01-19|Aerospatiale|SWITCHING HANDLE CONTROL DEVICE, PARTICULARLY FOR AIRCRAFT, AND SYSTEM COMPRISING SUCH A DEVICE|
    WO2006103774A1|2005-03-30|2006-10-05|Rikiya Ishikawa|Vertically movable flying body|
    US8078340B2|2007-11-12|2011-12-13|Honeywell International Inc.|Active user interface haptic feedback and linking control system using either force or position data|
    US20090302160A1|2008-06-04|2009-12-10|Eads Construcciones Aeronauticas, S.A.|System for refuelling operations|
    WO2010065036A1|2008-12-04|2010-06-10|The Boeing Company|Optimally positioning a refueling boom|
    WO2012030347A1|2010-09-02|2012-03-08|The Boeing Company|Imposed limits for limiting movement of a boom system|
    US9132921B2|2012-08-09|2015-09-15|The Boeing Company|Refueling boom control system|
    EP2738097B1|2012-11-29|2015-10-07|Airbus Defence and Space SA|In-Flight refueling systems with a digital communication sub-system|
    FR3005300B1|2013-05-03|2016-10-21|Eurocopter France|SYSTEM AND METHOD FOR CONTROLLING A TANGING STABILIZATION MEANS OF AN AIRCRAFT|
    SG2013039482A|2013-05-21|2014-12-30|Singapore Tech Aerospace Ltd|A system and method for transferring fuel in flight from a tanker aircraft to multiple receiver aircraft|
    CN103332299A|2013-07-09|2013-10-02|西北工业大学|Plane Z type stringer clamping device|
    EP2879312A1|2013-11-29|2015-06-03|EADS Construcciones Aeronauticas S.A.|A communication system for managing a flying operation involving two or more aircraft|
    US10132628B2|2015-03-06|2018-11-20|The Boeing Company|Aerial refueling boom elevation estimation system|CN112963248A|2021-02-22|2021-06-15|中国航发沈阳发动机研究所|System for filling lubricating oil in working state of aircraft engine|
    法律状态:
    2017-03-17| FG2A| Definitive protection|Ref document number: 2580252 Country of ref document: ES Kind code of ref document: B2 Effective date: 20170317 |
    优先权:
    申请号 | 申请日 | 专利标题
    ES201531426A|ES2580252B2|2015-10-05|2015-10-05|HAPPY INCREASED REALITY DEVICE TO FACILITATE ACTIONS ON A FLIGHT FILLING LOSS|ES201531426A| ES2580252B2|2015-10-05|2015-10-05|HAPPY INCREASED REALITY DEVICE TO FACILITATE ACTIONS ON A FLIGHT FILLING LOSS|
    US15/766,347| US11054931B2|2015-10-05|2016-10-04|Haptic augmented reality device for facilitating actions on a flying boom|
    PCT/ES2016/070701| WO2017051056A1|2015-10-05|2016-10-04|Haptic augmented reality device for facilitating actions on a flying boom|
    CN201680058780.2A| CN108351686B|2015-10-05|2016-10-04|Touch sense augmented reality device convenient to fly purlin operation|
    EP16848197.6A| EP3361349B1|2015-10-05|2016-10-04|Haptic augmented reality device for facilitating actions on a flying boom|
    AU2016328885A| AU2016328885A1|2015-10-05|2016-10-04|Haptic augmented reality device for facilitating actions on a flying boom|
    [返回顶部]